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Abstract 

Objective:  Determine  the  feasibility  of  using  routinely  collected  hospital  data  for 
medical  injury  surveillance.  Methods:  The  development,  validation,  and  testing  of 
screening  criteria  for  medical  injury  was  based  on  International  Classification  of 
Disease  code  discharge  diagnoses,  using  2001  patient  data  from  Wisconsin 
hospitals.  Outcomes  included  sensitivity;  specificity;  rate  of  medical  injury  per 
the  criteria;  and  impact  of  injury  on  length  of  stay  and  hospital  charges.  Results: 
Compared  with  medical  records  review,  the  sensitivity  of  the  screening  criteria 
was  59.9  percent  and  the  specificity  was  97.4  percent.  The  rate  of  medical  injury 
was  133.3  per  1,000  hospitalizations.  Patients  with  a  medical  injury  had  a  14.6 
percent  longer  hospital  stay  and  incurred  18.5  percent  more  in  hospital  charges 
than  patients  without  a  medical  injury.  Conclusions:  Screening  criteria  applied  to 
discharge  diagnoses  identify  frequently  occurring  medical  injuries  with 
substantial  impact.  These  criteria  are  being  used  to  monitor  patterns  of  medical 
injury  in  select  Wisconsin  hospitals  to  better  determine  the  utility  of  using 
hospital  discharge  data  to  focus  patient  safety  efforts. 


Introduction 

A  report  from  the  Institute  of  Medicine  (IOM)  in  2000,  To  Err  Is  Human,1 
focused  public  attention  on  the  issue  of  patient  safety  and  highlighted  the  need  for 
better  data  on  adverse  event  occurrences.  The  IOM  report  called  for  evaluations 
of  alternative  approaches  for  implementing  patient  safety  reporting  systems. 

While  the  IOM  report  primarily  emphasized  error  reporting  systems,  focusing  on 
the  injuries  or  adverse  events  that  result  from  medical  care  (rather  than  errors  of 

negligence)  may  be  more  reliable,  engender  less  defensiveness,  and  promote  a 

2 

greater  emphasis  on  patient  outcomes. 

Leape  recently  summarized  existing  national-  and  state-based  systems  for 
reporting  adverse  events.  National  systems  are  voluntary  and  most  focus  on 
relatively  few  types  of  reportable  events,  such  as  adverse  drug  events  or 
nosocomial  infections.  Twenty  states  have  developed  mandatory  patient  safety 
reporting  systems,  but  these  systems  vary  widely  by  the  types  of  events  to  be 


Because  a  patient  can  have  medical  injuries  in  more  than  one  category,  the  sum  of  injured  patients  in  each 
category  is  greater  than  the  total  number  of  patients  (see  Table  2). 
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reported  and  whether  the  reported  data  is  disseminated  to  the  public.  Most  of  the 
reports  made  to  these  systems  are  not  investigated  or  analyzed  in  detail. 

Surveillance  is  a  cornerstone  of  the  public  health  approach  to  disease  and 
injury  prevention.4, 5  A  core  principle  in  the  design  of  surveillance  systems 
maintains  that  any  system  requiring  special  effort  on  the  part  of  the  individual 
making  the  report  will  result  in  incomplete  reporting.  Therefore,  surveillance 
systems  that  use  information  collected  for  other  purposes  tend  to  provide  a  more 
complete  description  of  the  adverse  event.5  All  of  the  current  national-  and  state- 
based  systems  described  by  Leape3  require  some  active  effort  on  the  part  of  an 
individual  to  fde  a  report.  Not  surprisingly,  most  of  these  systems  have  produced 
adverse  event  reports  at  rates  substantially  lower  than  indepth  patient  safety 
studies  based  on  a  detailed  review  of  medical  records.  ’  ’  The  one  active 
surveillance  system  that  appears  to  have  a  high  level  of  ascertainment  of  adverse 
events  is  the  Centers  for  Disease  Control’s  (CDC)  National  Nosocomial  Infection 
Surveillance  System,  which  is  based  on  reports  from  hospital  infection  control 
practitioners,  who  participate  voluntarily. 

A  potential  source  of  routinely  collected  patient  safety  surveillance 
information  is  hospital  discharge  data.  Many  state  health  agencies  routinely 
collect  summary  data  for  all  inpatients  discharged  from  non-Federal,  acute  care 
hospitals  using  information  taken  from  the  Health  Care  Finance  Administration 
Uniform  Billing  Report  Form  (UB-92).  The  UB-92  data  includes  International 
Classification  of  Disease9  (ICD-9-CM)  A-codes  and,  where  relevant,  C-codes.  In 
cases  of  injury,  A-codes  attribute  the  injury  to  an  external  cause,  such  as 
therapeutic  misadventure.  A-codes  describe  the  nature  of  the  injury,  such  as  a 
complication  particular  to  a  specific  surgical  procedure,  but  make  no  attempt  to 
attribute  the  cause  of  the  injury. 

Because  they  are  collected  primarily  for  administrative  purposes,  hospital 
discharge  data  have  important  limitations  in  the  depth  of  clinical  detail  they 
provide.  At  the  same  time,  they  are  universally  available  in  many  states  and  are 
inexpensive  when  compared  with  an  active  reporting  system.  A  key  determinant 
of  the  utility  of  hospital  discharge  data  for  patient  safety  reporting  is  how 
effectively  adverse  events  can  be  identified  from  ICD-9-CM  A-code  and  A-code 
diagnoses. 

There  are  two  systems  relevant  to  patient  safety  reporting  that  make  use  of 
hospital  discharge  data:  the  Complications  Screening  Program  (CSP)10  and  the 
Agency  for  Healthcare  Research  and  Quality  -  Patient  Safety  Indicators  (AHRQ- 
PSI). 1 K  12  Both  of  these  systems  focus  on  assessing  the  provided  quality  of  care 
during  a  specific  inpatient  hospitalization.  Accordingly,  they  attempt  to  exclude 
patient  safety  problems  that  occur  prior  to  the  index  hospitalization,  such  as 
outpatient  care  or  care  provided  during  a  previous  hospitalization.  Each  of  the 
systems  also  focus  on  a  relatively  small  number  of  indicators  designed  to  reflect  a 
high  probability  of  poor  care  quality  during  the  hospitalization.  In  contrast,  major 
studies  based  on  chart  reviews  have  attempted  to  provide  a  more  comprehensive 
assessment  of  patient  safety  problems  by  including  the  full  spectrum  of  adverse 
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events  (i.e.,  those  that  occurred  during  the  index  hospitalization,  as  well  as  those 

6  7  8 

caused  by  medical  care  prior  to  the  index  hospitalization).  ’  ’ 

In  this  report,  we  describe  the  development  of  comprehensive  surveillance 
screening  criteria  for  medical  injury  based  on  ICD-9-CM  diagnostic  codes,9  and 
assess  the  accuracy  of  the  criteria  in  comparison  with  the  criterion  standard  of 
medical  record  review.  To  evaluate  the  utility  of  the  criteria  for  medical  injury 
surveillance,  we  applied  the  screening  criteria  to  a  routinely  collected 
administrative  database  of  all  patients  discharged  from  acute-care,  non-Federal 
Wisconsin  hospitals  during  2001. 


Methods 

Development  of  medical  injury  surveillance  criteria 

For  the  purpose  of  this  patient  safety  surveillance  system,  medical  injury  has 
been  defined  as  any  untoward  harm  associated  with  a  therapeutic  or  diagnostic 
health  care  intervention.  The  qualifier  “untoward”  excludes  damage  to  the  body 
that  is  an  inextricable  component  of  the  intervention,  such  as  a  surgical  incision. 
To  develop  comprehensive  surveillance  or  screening  criteria  for  medical  injuries, 
we  reviewed  the  ICD-9-CM  codes  to  identify  A-codes  or  E-codes  indicative  of  a 
medical  injury.  An  electronic  version  of  the  1998  ICD-9-CM  update  was  obtained 
to  facilitate  identification  of  potential  medical  injury  codes  through  searches  for 
keywords  such  as  “complications,”  “iatrogenic,”  “poisoning,”  and  “adverse.”  In 
addition,  a  physician  reviewed  each  rubric  in  the  ICD-9-CM  to  identify  potential 
medical  injuries  not  associated  with  any  of  the  keywords.  Electronic  lists  of  ICD- 
9-CM  codes  to  be  included  and  excluded  from  the  medical  injury  surveillance 
criteria  then  were  circulated  to  other  members  of  the  research  team,  including 
physicians,  nurses,  statisticians,  epidemiologists,  and  a  medical  technologist. 
Additional  codes  were  made  part  of  the  surveillance  criteria  on  the  basis  of  this 
review. 

Medical  injuries  were  classified  into  four  broad  categories:  (1)  drugs;  (2) 
procedures;  (3)  devices,  implants,  or  grafts;  and  (4)  radiation.  These  categories 
were  further  divided  into  40  subcategories  to  indicate  the  cause  of  injury  more 
precisely.  The  classification  scheme  and  the  ICD-9-CM  criteria  for  each  of  the 
categories  and  subcategories  are  listed  in  the  Appendix. 

Case  definition 

In  this  study,  a  discharge  was  considered  to  fulfill  the  criteria  for  medical 
injury  if  any  of  the  nine  diagnosis  fields  or  the  special  E-code  field  contained  any 
one  of  the  codes  listed  in  the  surveillance  criteria.  A  given  patient  discharge  could 
be  associated  with  more  than  one  type  of  medical  injury. 


Appendixes  cited  in  this  report  are  provided  electronically  at  www.mcw.edu/medicalinjury. 
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Validation 

To  validate  the  medical  injury  surveillance  criteria,  we  compared 
identification  of  medical  injury  based  on  hospital  discharge  diagnoses  to  the  “gold 
standard”  of  a  detailed  medical  records  review.  Medical  records  of  100  patients 
discharged  in  1999  from  a  single  teaching  hospital  were  included  in  the  validation 
study.  A  random  sample  of  50  medical  records  for  patients  who  met  the 
surveillance  criteria  based  on  discharge  diagnoses,  and  50  records  of  patients  who 
did  not  meet  the  criteria  were  identified  and  subjected  to  a  blinded  review.  An 
experienced  research  nurse  abstracted  the  relevant  data  from  the  records.  Two 
physicians  then  rated  each  patient  for  the  occurrence  of  any  medical  injury,  based 
on  the  abstracted  data  and  without  any  knowledge  of  the  injury  classification 
developed  from  the  screening  criteria.  In  the  event  of  a  question,  the  physicians 
reviewed  the  entire  medical  record  for  the  patient  admission.  Disagreements  were 
resolved  by  a  third  physician,  used  in  only  three  cases.  The  reviewers’  response  to 
the  question,  “Did  this  patient  experience  untoward  harm  as  a  result  of  a 
diagnostic  or  therapeutic  healthcare  intervention?”  was  the  criterion  used  to 
produce  the  gold  standard  classification  of  medical  injury  for  the  medical  records 
review.  Sensitivity  and  specificity  (and  the  95  percent  confidence  intervals)  were 
adjusted  for  the  different  proportions  of  medical  records  reviewed,  among  those 

13 

patients  who  met  the  surveillance  criteria  and  those  who  did  not. 

Study  population 

The  Wisconsin  Bureau  of  Health  Information  (BHI)  collects,  edits,  and 
publicly  disseminates  Form  UB-92  data  on  all  patients  discharged  from  non- 
Federal  hospitals  in  the  state.  To  better  assess  the  utility  of  the  surveillance 
criteria  in  identifying  medical  injuries,  we  studied  discharges  from  Wisconsin’s 
132  general,  acute  care  hospitals,  during  2001.  Newborn  delivery  discharges  were 
excluded  for  the  purposes  of  this  analysis. 

Statistical  analysis 

Rates  of  medical  injuries  were  calculated  as  the  number  of  discharges  with  the 
particular  type  of  medical  injury,  divided  by  the  total  number  of  discharges. 
Statistical  analysis  was  performed  using  S tala  ''  software.14  Rate  ratios  and  95 
percent  confidence  intervals  for  specific  discharge  outcomes  in  patients  with  a 
medical  injury  compared  to  those  without  injuries  were  calculated  using  standard 
methods  for  categorical  data.14 

The  impact  of  medical  injury  on  length  of  hospital  stay  (LOS)  and  on  hospital 
charges  was  calculated  using  linear  regression  on  log  transformations  of  the  LOS 
and  charge  data  for  patient  discharges  with  and  without  medical  injury.  For 
analytic  purposes,  we  added  one  day  to  the  routinely  reported  length  of  stay  so 
that  the  measure  indicates  the  number  of  days  for  which  the  patient  was 
hospitalized.  Therefore,  patients  admitted  and  discharged  on  the  same  day  were 
considered  to  have  had  a  one  day  length  of  stay.  Mean  excess  LOS  and  percent  of 
LOS  increase  compared  to  discharges  without  the  specified  medical  injury  were 
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determined  after  adjusting  for  APR-DRG  (All  Patient  Refined-Diagnosis  Related 
Groups)  disease  classification,15  indices  for  risk  of  mortality,  and  severity  of 
illness  calculated  by  the  APR-DRG  medical  information  system.  Because  the 
indices  for  risk  of  mortality  and  severity  of  illness  were  intended  to  adjust  for  the 
severity  of  the  underlying  illness  in  patients  with  and  without  a  medical  injury, 
they  were  calculated  after  excluding  all  medical  injury-related  diagnostic  codes. 
“Hospital  of  discharge”  also  was  used  as  a  random  effect  variable  in  the 
regression  model,  to  account  for  intrahospital  similarities.  Predicted  cumulative 
excess  LOS  for  each  category  of  medical  injury  was  calculated  as  the  product  of 
the  mean  excess  LOS  and  the  number  of  cases  in  the  specific  category.  Less  than 
0.2  percent  of  patients  could  not  be  assigned  a  severity  of  underlying  illness  score, 
primarily  because  after  excluding  these  medical  injury  codes,  no  valid  codes 
representative  of  the  principal  diagnosis  remained.  Those  patients  who  could  not 
be  assigned  a  severity  of  illness  score  were  dropped  from  analyses  of  the  length  of 
hospital  stay.  In  analyses  of  hospital  charge  data,  those  patients  who  could  not  be 
assigned  a  severity  of  illness  score,  who  had  a  principal  diagnosis  of  “V650” 
(Healthy  person  accompanying  sick  person),  or  who  had  no  hospital  charges  were 
dropped.  This  study  methodology  was  reviewed  and  approved  by  the  Medical 
College  of  Wisconsin’s  Institutional  Review  Board. 


Results 

Validation 

The  medical  records  review  confirmed  the  presence  of  a  medical  injury  in  45 
of  the  50  patients  (90  percent)  who  met  the  surveillance  criteria,  and  determined 
that  7  of  the  50  patients  (14  percent)  who  did  not  meet  the  screening  criteria  had, 
in  fact,  experienced  a  medical  injury  (Table  1).  Using  the  chart  review  as  the  gold 
standard  for  medical  injury  identification,  the  screening  criteria  based  on 
discharge  diagnoses  had  a  sampling-adjusted  sensitivity  of  59.9  percent 
(95  percent  confidence  interval:  42.8  percent  to  75.0  percent)  and  specificity  of 
97.4  percent  (95  percent  confidence  interval:  94.1  percent  to  98.8  percent). 

Table  1.  Validation  of  medical  injury  surveillance  criteria 


Medical  chart  review 


Injury 

No 

Injury 

Total 

Screening 

criteria 

Injury 

45 

5 

50 

No  Injury 

7 

43 

50 

Total 

52 

48 

Sensitivity*  =  59.9%  (95%  confidence  interval:  42.8%-  75.0%) 
Specificity*  =  97.4%  (95%  confidence  interval:  94.1%-  98.8%) 
*  Adjusted  for  selection  of  the  verification  sample 
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Incidence  and  nature  of  medical  injuries 

After  applying  the  screening  criteria  to  the  558,389  eligible  discharges  from 
the  selected  Wisconsin  acute  care  hospitals,  we  found  that  74,447  patients 
experienced  a  medical  injury — a  rate  of  133.3  per  1,000  hospital  discharges 
(Table  2).  The  surveillance  criteria  flagged  more  than  33,000  discharges  with  a 
patient  injury  from  a  medical  or  surgical  procedure;  more  than  31,000  discharges 
with  injuries  due  to  a  drug  or  biologic  agent;  more  than  16,000  discharges  with 
injuries  due  to  failure  or  complications  of  a  device,  implant,  or  graft;  and  more 
than  1,400  discharges  with  injuries  due  to  radiation. 


Table  2.  Rate  of  medical  injury  in  patients  discharged  from  Wisconsin  general,  acute 
care  hospitals,  by  category,  2001 


Category 

Cases 

Rate/1000 

discharges 

Mean  excess 
LOS  (days)* 

Percent 

increased 

LOS* 

Cumulative 

excess 

LOS** 

Drug 

31,382 

56.2 

0.47 

8.6 

14,611 

Device, 
implant,  or 
graft 

16,267 

29.1 

0.87 

16.0 

14,093 

Procedure 

33,225 

59.5 

1.10 

20.5 

36,710 

Radiation 

1,469 

2.6 

0.69 

12.7 

1,015 

Any  injury 

74,447 

133.3 

0.77 

14.5 

57,619 

*  Mean  excess  length  of  stay  (LOS)  and  percent  increased  LOS  compared  to  discharges  without 
the  specified  medical  injury  after  adjusting  for  APR-DRG  (All  Patient  Refined  -  Diagnosis  Related 
Groups),  risk  of  mortality  and  severity  of  illness  (see  Methods  section). 

**  Cumulative  excess  LOS  is  the  product  of  cases  and  mean  excess  LOS. 

Note:  The  number  of  injured  patients  in  each  category  sum  to  more  than  the  total  number  of 
injured  patients  because  patients  can  have  a  medical  injury  in  more  than  one  category. 


A  more  detailed  categorization  of  the  nature  of  the  medical  injuries  is 
provided  by  the  40  subcategories  shown  in  Table  3.  The  table  provides  the 
number  and  rate  of  occurrence  in  each  category,  as  well  as  the  excess  length  of 
hospital  stay  (in  days)  associated  with  each  subcategory. 

Both  E- codes  and  A-codes  were  useful  for  identifying  medical  injuries.  Of  the 
74,447  patient  discharges  with  medical  injuries,  22,586  (30.3  percent)  were 
identified  only  through  an  E- code;  31,117  (41.8  percent)  were  identified  only 
through  an  A-code;  and  20,744  (27.9  percent)  had  both  E- code  and  A-code 
diagnoses  indicative  of  the  injury.  The  relative  utility  of  A-codes  and  E- codes 
varied  substantially  among  the  different  types  of  medical  injuries.  E- codes 
identified  98.5  percent  of  the  medical  injuries  due  to  drugs  and  78.9  percent  of 
those  due  to  radiation,  but  only  22.5  percent  of  injuries  due  to  medical  or  surgical 
procedures  and  26.1  percent  of  those  linked  to  devices,  implants,  or  grafts. 
Conversely,  A-codes  identified  only  22.1  percent  of  medical  injuries  due  to  drugs 
and  53.9  percent  of  those  due  to  radiation,  but  95.5  percent  of  injuries  due  to 
medical  or  surgical  procedures  and  89.3  percent  of  those  associated  with  devices, 
implants,  or  grafts. 
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Table  3.  Rate  of  medical  injury  in  patients  discharged  from  Wisconsin  general  acute 
care  hospitals,  by  subcategory,  2001 


Subcategory 

Cases 

Rate/1,000 

Discharges 

Mean  Excess 
LOST  (days) 

DRUGS  and  BIOLOGICS 

Antibiotics 

2,745 

4.9 

1  27*** 

Hormones 

3,580 

6.4 

0.70*** 

Systemic  agents 

3,825 

6.9 

0.47*** 

Agents  affecting  blood  constituents 

1,887 

3.4 

0.77*** 

Blood  products 

401 

0.7 

0.82*** 

Non-narcotic  analgesics, 
antipyretics,  antirheumatics 

4,027 

7.2 

N/A| 

Narcotic  analgeics 

2,483 

4.4 

0.38*** 

Anticonvulsants,  antiparkinsonism 

1,206 

2.2 

0.36*** 

Sedatives,  hypnotics 

2,410 

4.3 

N/At 

Other  psychotropics 

2,951 

5.3 

N/A| 

Autonomic  nervous  system 

591 

1.1 

N/At 

CV  drugs 

3,637 

6.5 

0.47*** 

Gl,  smooth  muscle  and  respiratory 

581 

1.0 

N/At 

Water,  mineral,  uric  acid 

1,772 

3.2 

0.41*** 

Anesthesia 

540 

1.0 

N/At 

Other  drug  complications 

538 

1.0 

N/At 

Specific  reaction  to  unknown  drug 

1,316 

2.4 

1 .43*** 

Misc.  comp  due  to  unspecified  drugs 

2,923 

5.2 

0.02** 

All  drug-related  injuries 

31,382 

56.2 

0.47** 

DEVICE,  IMPLANT,  OR  GRAFT 

Cardiac  device 

1,243 

2.2 

0.15* 

Vascular  device 

4,630 

8.3 

1.23*** 

Orthopedic  device 

3,892 

7.0 

0.20*** 

GU  device  or  implant 

1,518 

2.7 

1.01*** 

Transplanted  organ  or  body  part 

1,791 

3.2 

1.37*** 

Other  comp  device 

5,321 

9.5 

0.68*** 

All  device,  implant,  and  graft-related 
injuries 

16,267 

29.1 

0.87*** 
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Table  3.  Rate  of  medical  injury  in  patients  discharged  from  Wisconsin  general  acute 
care  hospitals,  by  subcategory,  2001,  cont. 


Subcategory 

Cases 

Rate/1,000 

Discharges 

Mean  Excess 
LOST  (days) 

PROCEDURE 

Comp  of  amputation  or  other 
removed  organ 

2,017 

3.6 

1.35*** 

Tracheostomy 

263 

0.5 

0.60*** 

Formation  or  a  stoma 

1,114 

2.0 

1.03*** 

Gl  comp  due  to  procedure 

4,554 

8.2 

1.95*** 

Cardiac  complications 

3,470 

6.2 

0.49*** 

Vascular  complications 

2,361 

4.2 

0.61*** 

Respiratory  complications 

3,771 

6.8 

1.14*** 

Nervous  system  comp 

1,489 

2.7 

1.37*** 

GU  complications 

1,956 

3.5 

0.86*** 

Infection/inflammation 

4,203 

7.5 

2.01*** 

Hematoma,  hemorrhage,  seron 

4,958 

8.9 

1.06*** 

Nonhealing  wound 

3,657 

6.5 

1  22*** 

Misc.  comp  due  to  specific 
procedures 

1,080 

1.9 

0.24*** 

Misadventures 

2,773 

5.0 

0.84*** 

Not  elsewhere  specified 
complications 

6,413 

11.5 

0.88*** 

All  procedure-related  injuries 

33,225 

59.5 

1.10*** 

RADIATION 

Radiation 

1469 

2.6 

0.69*** 

All  radiation-related  Injuries 

1469 

2.6 

0.69*** 

Total  all  medical  injuries 

74,447 

133.3 

0.69*** 

Source:  Wisconsin  Bureau  of  Health  Information;  hospital  inpatient  discharge  data  of  year  2001 
t  Mean  Excess  Length  of  Stay  (LOS)  for  each  subcategory  of  medical  injury  compared  to 
discharges  without  the  specified  medical  injury  were  calculated  after  adjusting  for:  APR-DRG  (All 
Patient  Refined-Diagnosis  Related  Groups),  risk  of  mortality  and  severity  of  illness — see 
methods. 

f  N/A  No  increased  LOS  was  associated  with  the  subcategory. 

*  P  <  0.05 
**  P<  0.01 
***  P<  0.001 

Impact  and  outcomes 

Analysis  of  the  BHI  data  indicated  that  medical  injuries  were  associated  with 
a  substantial  impact  among  those  affected.  Compared  with  those  without  any  such 
diagnosis,  patients  with  a  discharge  diagnosis  indicating  a  medical  injury  stayed 
in  hospitals  an  average  of  0.77  days  longer  after  adjusting  for  APR-DRG,  severity 


126 


Injury  Identification  and  Discharge  Data 


Table  4.  Adjusted  length  of  stay  and  hospital  charges  by  diagnosis  of  Wisconsin 
medical  injury  inpatients,  for  2001*** 


Patients  with 
medical  injury 

Patients 
without 
medical  injury 

Absolute 

difference 

Percent 

difference*** 

95% 

confidence 

limits 

Adjusted 
length  of 
hospital 
stay  (days)* 

6.11 

5.34 

0.77 

14.5% 

14.1%- 

15.0% 

Adjusted  total 

hospital 

charges** 

$14,916 

$12,587 

$2,329 

18.5% 

18.0%- 

19.0% 

*  Sample  size  for  patients  with  medical  injury  is  73,485  and  for  patients  without  medical  injury  is 
483,840. 

**  Sample  size  for  patients  with  medical  injury  is  73,483,  and  for  patients  without  medical  injury  is 
483,819. 

***  Adjusted  for  severity  of  underlying  illness,  risk  of  mortality,  APR-DRG  (see  “Methods”  section). 

of  illness,  risk  of  mortality,  and  hospital  of  discharge;  they  also  incurred  $2,329  in 
additional  hospital  charges  (Table  4). 


Discussion 

Surveillance  of  medical  injuries  through  analysis  of  routinely  collected 
hospital  discharge  data  appears  feasible.  Our  validation  study  found  that  screening 
criteria  based  on  ICD-9-CM  rV-code  and  ii-code  discharge  diagnoses  can  identify 
medical  injuries  with  good  sensitivity  and  specificity,  compared  with  the  gold 
standard  of  medical  record  review.  Application  of  the  surveillance  criteria  to  all 
Wisconsin  hospital  discharges  during  2001  indicates  that  the  framework  provides 
an  efficient  approach  to  patient  safety  monitoring  that  can  identify  a  diverse  range 
of  medical  injuries.  The  increase  in  hospital  charges  and  the  length  of  hospital 
stay  associated  with  these  injuries  point  to  their  economic  and  clinical  importance. 
Moreover,  projecting  the  rate  of  medical  injury  found  in  this  study  to  the  31.7 
million  discharges  from  U.S.  acute  care  hospitals  during  2000, 16  suggests  that 
more  than  3.5  million  patients  are  discharged  annually  with  a  diagnosis  indicative 
of  a  medical  injury. 

The  need  for  better  approaches  to  monitoring  patient  safety  was  highlighted  in 
the  Institute  of  Medicine  report  To  Err  Is  Human. 1  That  report  called  for 
mandatory  reporting  of  serious  patient  safety  incidents  and  voluntary  reporting  of 
other  errors  or  incidents.  Most  current  patient  safety  reporting  systems  focus  on 
incidence  of  medical  error  or  negligence.  Perceived  blame  and  punishment  for 
error,  however,  may  be  an  incentive  for  concealment  and  denial.  Barriers  to  the 
reporting  of  errors  or  negligence  include  fear  of  possible  legal  discovery  in 
malpractice  litigation  and  adverse  publicity,  which  could  have  economic 
consequences  as  severe  as  those  resulting  from  malpractice  damages. “  In  addition, 
the  determination  of  negligence  or  medical  error  often  is  arbitrary.  Interobserver 
agreement  in  identifying  negligence,  for  example,  appears  to  be  substantially 
lower  than  that  for  identifying  injury  caused  by  health  care  interventions.  ’  The 
criteria  evaluated  in  this  report  focus  on  the  identification  of  medical  injury — 
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untoward  harm  resulting  from  a  diagnostic  or  therapeutic  health  care 
intervention — rather  than  on  error  or  negligence.  We  believe  this  injury 
prevention  approach  is  a  useful  complement  to  other  approaches  that  focus  on 
error  reduction. 

Health  care  facilities  in  Wisconsin  are  required  to  submit  inpatient  discharge 
data  files  and  ambulatory  surgery  data  files  to  BHI.  Similar  datasets  exist  in  a 
number  of  other  states,  which  would  permit  future  application  of  this  patient 
safety  reporting  system  model  beyond  Wisconsin.  For  example,  AHRQ’s 
Healthcare  Cost  and  Utilization  Project  (HCUP)  compiles  similar  patient-level 
discharge  data  in  its  Nationwide  Inpatient  Sample  (NIS)  and  State  Inpatient 
Databases  (SID).  These  datasets  extend  the  capacity  for  useful  comparisons  and 
contrasts. 

The  application  of  these  medical  injury  screening  criteria  to  large  population- 
based  databases  permits  public  health  surveillance  of  patient  safety.  Potential  uses 
of  public  health  surveillance  systems  have  been  delineated5  to  include  estimating 
the  magnitude  of  a  health  problem  in  the  population;  understanding  the  natural 
history  of  disease  or  injury;  documenting  the  distribution  and  spread  of  health 
events;  testing  hypotheses  concerning  etiology;  evaluating  control  strategies; 
detecting  change  in  specific  health  practices;  identifying  research  needs; 
contributing  to  epidemiologic  and  laboratory  research;  and  facilitating  planning. 
These  potential  applications  of  surveillance  systems  all  are  relevant  to  the  nascent 
field  of  patient  safety. 

Our  criteria  formed  the  basis  of  a  randomized  trial  using  Wisconsin  hospitals 
to  evaluate  the  utility  of  medical  injury  feedback  reports,  and  the  targeted 
interventions.  Each  of  the  129  Wisconsin  general  acute-care,  non-Federal 
hospitals  that  were  in  operation  at  the  time  of  randomization  were  assigned  to  one 
of  three  groups:  control  (34),  feedback  report  only  (65),  and  intervention  (30).  In 
addition  to  the  feedback  reports,  the  intervention  group  had  access  to  a  nurse  who 
assisted  in  interpreting  the  reports  and  implementing  patient  safety  efforts,  and  the 
group  received  manuals  detailing  evidence-based  methods  for  reducing  five 
specific  medical  injuries.  The  reports  have  been  provided  at  approximately  6 
month  intervals  and  cover  consecutive  6-month  time  periods.  They  contain  four 
tables:  (1)  number  of  occurrences,  adjusted  injury  rates  per  1000  discharges, 
percentile  ranking  among  Wisconsin  hospitals,  and  statewide  increase  in  length  of 
stay,  for  the  4  broad  categories;  (2)  similar  details  for  the  40  additional 
subcategories  of  medical  injuries;  (3)  the  10  most  frequent  events  at  the  institution 
with  the  greatest  increase  in  length  of  stay;  and  (4)  injury  rates  of  16 
subcategories  relevant  to  surgery,  using  only  surgical  discharges  in  the  calculation 
of  rates.  We  are  currently  refining  risk  adjustment  methods  and  exploring 
performance  measures  based  on  multilevel  modeling  for  our  administrative 
surveillance  data.  An  evaluation  of  the  impact  of  the  feedback  reports  is 
underway. 

Few  comprehensive  studies  of  patient  safety,  adverse  events,  or  medical 
injuries  are  available  to  provide  a  comparison  with  our  results.  A  landmark  study 
in  the  patient  safety  field,  the  Harvard  Medical  Practice  Study,  defined  adverse 
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events  as,  “injuries  caused  by  medical  management  and  that  prolonged 

/:  17 

hospitalization,  produced  a  disability  at  the  time  of  discharge  or  both.”  ’  Studies 
in  Australia,  and  in  Colorado  and  Utah,  employed  similar  definitions  and  included 
detailed  reviews  of  thousands  of  medical  records.  ’  ’  Despite  the  similarity  of 
methods,  however,  the  estimates  of  adverse  events  in  these  studies  range  from 
2.9  percent  to  16.6  percent  (Table  5).  Our  estimate  of  the  overall  rate  of  medical 
injury  is  within  that  range.  Our  screening  criteria  identified  a  higher  proportion  of 
patient  safety  problems  due  to  drugs  than  in  the  chart  based  review  studies,  but  a 
similar  proportion  due  to  operations  or  procedures. 

In  contrast  to  the  comprehensive  studies  of  patient  safety,  and  our  surveillance 
approach,  the  AHRQ  Patient  Safety  Indicators11' 12' 19  and  the  Complications 
Screening  Program10' 20  focus  exclusively  on  in-hospital  patient  safety  problems. 
As  a  result,  the  AHRQ  Patient  Safety  Indicators  and  the  Complications  Screening 
Program  are  more  appropriate  tools  for  evaluating  the  quality  of  care  in  hospitals. 
Our  approach,  in  contrast,  includes  medical  injuries  identified  or  treated  in- 
hospital — even  if  they  occurred  in  the  outpatient  setting,  a  nursing  home,  or 
during  previous  hospitalizations.  It  provides  a  more  accurate  estimate  of  the  full 
scope  of  patient  safety  problems.  In  addition,  it  can  be  used  to  target  certain 
patient  safety  problems  such  as  drug  reactions  and  device  failures,  which  occur 
frequently  in  an  outpatient  setting. 


Table  5.  Rates  of  adverse  events  or  medical  injury  in  selected  studies 


Study* 

HMPS 

QAHCS 

CUS 

WMIPP 

Location 

New  York 

New  South 
Wales,  South 
Australia 

Colorado  and 
Utah 

Wisconsin 

Year 

1984 

1994 

1992 

2001 

Ascertainment 
of  adverse 
events/ 
medical  injury 

Medical 
record  review 

Medical  record 
review 

Medical  record 
review 

Discharge 

diagnosis 

screening 

Number  of 
patients 

30,121 

14,179 

15,000 

581,104 

Adverse  event 
or  medical  injury 
rate  / 100 
discharges 

3.7 

16.6 

2.9 

13.3 

%  of  Adverse 
event  or  medical 
injury  due  to 
operations  or 
procedures 

48.0% 

50.3% 

44.9% 

40.3% 

%  of  Adverse 
event  or  medical 
injury  due  to 
drugs 

19.0% 

10.0% 

19.0% 

38.1% 

*  HMPS  =  Harvard  Medical  Practice  Study;  QAHCS  =  Quality  in  Australian  Healthcare  Study; 
CUS  =  Colorado/Utah  Study;  WMIPP  =  Wisconsin  Medical  Injury  Prevention  Program 
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Limitations  of  this  approach  to  patient  safety  monitoring  should  be 
considered.  Because  it  is  based  on  hospital  discharge  data  collected  for 
administrative  purposes,  this  approach  shares  the  same  limitations  of  all  studies 
that  rely  on  such  data.'  The  validation  study  described  in  this  report  was 
performed  using  a  sample  of  medical  records  from  a  large,  academic  hospital  with 
the  staff  and  resources  to  commit  to  the  detailed  coding  of  discharge  diagnoses. 
Completeness  of  medical  records  and  specificity  of  coding  in  other  hospitals  may 
influence  the  ability  of  the  criteria  to  detect  medical  injuries;  accordingly,  the 
criteria  should  be  revalidated  in  diverse  hospital  settings.  While  medical  record 
review  is  regarded  as  the  gold  standard  in  patient  safety  studies,  there  is  evidence 
of  variability  among  reviewers  in  their  determination  of  adverse  events  and 
negligence.  ’  Studies  have  found,  however,  that  reliability  is  higher  for  the 
detection  of  adverse  events  (which  is  analogous  to  our  definition  of  medical 
injury)  than  for  negligence  or  error."  Another  limitation  to  our  approach  is  the 
variability  among  states  in  external  cause  of  injury  coding,  with  respect  to 
hospital  discharge  data.  In  Wisconsin,  however,  external  cause  of  injury  coding 
is  complete;24  our  approach  would  be  of  greatest  utility  in  states  with  a  high  level 
of  fi-coding.  At  the  same  time,  the  face  validity  of  the  diagnostic  codes 
comprising  the  criteria,  the  agreement  of  the  criteria  with  medical  record  review, 
and  the  adverse  outcomes  experienced  by  individuals  fulfilling  the  criteria  all 
indicate  that  the  criteria  are  useful  in  identifying  injuries  from  health  care 
interventions  that  have  an  appreciable  impact  on  patient  health. 


Conclusion 

The  Wisconsin  Medical  Injury  Prevention  Program  Screening  Criteria  can 
identify  medical  injuries  from  hospital  discharge  data.  As  with  other  screening 
criteria,  sensitivity  and  specificity  are  imperfect  but  appear  to  be  sufficient  for 
monitoring  patterns  and  trends  in  medical  injuries — something  existing  systems 
are  not  equipped  to  do  well.  Using  these  criteria,  areas  most  in  need  of  patient 
safety  interventions  can  be  identified  and  targeted.  This  approach  provides  patient 
safety  researchers  with  another  useful  tool,  in  addition  to  those  studies  based  on 
detailed  chart  review  and  those  and  those  that  rely  on  ad  hoc  error  reporting 
systems.22 
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